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Abstract 



We present a developing distributed cognition theory of interdisciplinary collaboration that 
incorporates concepts from both situated cognition and information processing theory. This 
theoretical framework is being refined as it is used for analyzing interdisciplinary collaboration 
within the National Institute of Science Education (NISE). Our analyses are intended to improve 
scientific understanding of collaborative processes that influence productivity and quality of 
interdisciplinary work within the NISE and beyond. A critical group meeting in the early 
development of one interdisciplinary working team is analyzed using language and ideas from 
our theoretical perspective. 



A major strategy of the National Institute for Science Education (NISE) is to create 
interdisciplinary teams to work on the Institute's mission of improving science, mathematics, 
engineering, and technology (SMET) education. Within the Institute, experts from a variety of 
disciplines form working teams of various durations that study significant issues and propose and 
carry out projects related to NISE goals. Interdisciplinary teams are increasingly common in 
industry, government, and education and currently play an extremely important role in systemic 
educational reform. 1 While there exists a growing body of literature and much "wisdom of 
practice" on interdisciplinarity (e.g., Chubin, Porter, Rossini, & Connolly, 1986), there is limited 
scientific understanding of how group cognition drives intellectual growth and product 
development within natural interdisciplinary groups tackling important social issues. Toward the 
goal of advancing scientific knowledge about interdisciplinary collaboration, we are analyzing 
cases of teamwork within the NISE. Our analyses are intended to help understand, and ultimately 
design and implement new ways to facilitate, collaborative processes that influence productivity 
and quality of interdisciplinary work within the NISE and beyond. 

Theoretical Perspective 

Our developing framework for conceptualizing collaborative cognition is influenced by two 
theoretical viewpoints: (a) situated cognition theories (e.g., Brown, Collins, & Duguid, 1989; Lave, 
1991; Lave & Wenger, 1991) and (b) groups as information processors (Hinsz, Tindale, & 

Vollrath, 1997; O'Donnell, DuRussel, & Derry, 1997; Smith, 1994). Our understanding of each 
will be overviewed briefly and their integration in our work will be described. 

Situated Cognition 

From this perspective, working teams are viewed as living, evolving communities defined largely 
by their practices and the tools they use to carry out their practices (Lave, 1991; Brown, Collins, & 
Duguid, 1989). Practices refer to group activities as regulated, explicitly or implicitly, by rules or 
standards called norms. Tools include intangible tools, such as shared concepts and language 
conventions, as well as tangible and technological ones, such as computer systems, and are 
believed to play important roles in shaping a community’s thoughts and actions. 

Groups function within broader contexts — cultural, institutional, and physical — that form and 
constrain their development. In fact, most of a group’s thought-shaping tools will be cultural and 
institutional artifacts, which have evolutionary histories and may be strongly entrenched at a 
higher (than group) level. Also evolved and possessing history are establishment norms and 
institutional structures that influence and constrain group practice. To illustrate the shaping forces 
of such contexts, consider the typical case of a working group whose members have offices in 
different buildings. In this context the group might come to rely heavily on e-mail and periodic 
formal meetings for communication. Situated cognition would predict that this same group would 
develop significantly different practices if its members all worked in physical closeness to one 
another. For example, they might develop strong social ties and employ only informal, face-to-face 
communication. Another illustration is provided by the case in which a management change leads 
to an increased institutional emphasis on (for example) family values. Situated cognition would 
predict that the language and knowledge of working groups within that institution would 
increasingly reflect concern with this family values agenda. 
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From this perspective, both language and knowledge development in groups are driven by a 
process characterized by Lave (1991) and others as “negotiation.” Negotiation is necessary 
because different members bring to the group their own cognitive histories, and these unique 
perspectives cause members to understand and interpret work-related problems in significantly 
different ways. Negotiation describes communication processes that help align language and 
understandings among community members. Negotiation appears to start with team members’ 
finding “boundary concepts” (Wenger, 1990) or “common voices” (Wertsch, 1991). Both terms 
refer to finding perspectives and language that are already shared to some extent by team members 
from different disciplinary cultures. 

DuRussel and Derry (1996) described several ways in which members of an interdisciplinary team 
sought common voice during early stages of their teamwork. One method was to find viewpoints 
representing communities in which all or many participants already shared membership. For 
example, both physical and social scientists found common ground in their understanding of 
experimental control , a concept common to their mutual experiences as readers and designers of 
experimental research. Another was to draw explicit metaphorical links between ideas from two 
communities. For example, the astronomer’s understanding of galaxies was likened to the 
educator’s view of school systems. However, DuRussel and Derry described only early stages of 
team communication, and finding common ground is likely only an initial step. For as 
interdisciplinary work proceeds, groups must realize that shared concepts actually differ as their 
details are unpacked. This is because the practices in which those concepts are used are markedly 
different. Hence, the chemist’s and social scientist’s understanding of experimental control would 
differ at a more fine-grained level. Moreover, the interdisciplinary practice of the new, emerging 
group might require yet another understanding of that concept within the specific context of that 
group’s work. Hence, a specific conceptualization of experimental control would need to be 
negotiated in the context of interdisciplinary group work. In this sense, language and thought are 
believed to be situated by context. 

Another way that DuRussel and Derry’s (1996) interdisciplinary team found common voice 
resulted from the fact that one disciplinary perspective assumed greater status than the others. 
Unequal status favoring one discipline emerged because the research problem itself was associated 
with one discipline more than others, because the project director represented the dominant 
discipline, and because, relative to other viewpoints, the dominant perspective was shared by more 
members of the team. Thus early negotiation took the form of cognitive apprenticeship , whereby 
members associated with the dominant discipline mentored those who were unfamiliar with the 
rhetoric and practices of that discipline. A cognitive apprenticeship phase is in fact predicted by 
theories of situated cognition, which describe cognitive development as a process of gradually 
increasing participation within a community, participation facilitated through mentorship by “old- 
timers” (Lave, 1991). However, DuRussel and Derry argued that the classic cognitive 
apprenticeship model is a flawed one for interdisciplinary teamwork, since the desirable outcome 
of interdisciplinary problem solving should not be the assimilation of various perspectives by a 
single dominant one. If interdisciplinary research is to benefit from the diversity of perspectives 
brought to the table, all perspectives must have sufficient status so that n-way mentorships and 
apprenticeships would be necessary. 
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Information Processing Views of Collective Intelligence 

In a recent review of research on small-group processes, Hinsz et al. (1997) described an 
emerging view of groups as information processing systems. Their review represented a trend 
toward employing the classic information processing model of individual human cognition as an 
analogy for describing information processing in groups. We believe that a powerful theory of 
group processing will emerge by combining situated cognition theory with information 
processing concepts, which provide a language for explicitly describing how knowledge is 
negotiated in groups and transformed into products. 

Classic information processing models describe individual human cognition in terms of a long- 
term memory store; a short-term working memory; basic memory processes that filter, transform, 
combine, and store information; and executive processes that control memory operation (e.g., 
Hinsz et al., 1997; Smith, 1994). Although long-term memory capacity is virtually unlimited, all 
information input to it must pass through limited capacity attentional and other working memory 
processes that serve as a bottleneck for acquiring new knowledge. The system works as follows: 

At any point in time, a small portion of one’s long-term memory network, or prior knowledge, is 
currently active and hence accessible within one’s immediate span of attention, the working 
memory. Active prior knowledge provides a mental context that influences the filtering, 
organizing, and interpreting of incoming environmental information, which also passes through 
working memory. As prior knowledge and new information are attended to and integrated 
together, the active long-term memory structures are updated by new information. In this way, 
new knowledge is acquired as a byproduct of a process whereby new information is interpreted 
in terms of prior knowledge. The human information processing system has some capacity to 
manage this activity at all levels, a capability called “executive” or “metacognitive” control 
(Derry, 1992). 

Like situated cognition theory, information processing views provide an important role for tools. 
Both long-term and short-term memory stores can be augmented and extended through tool use. 
For example, the hard disk storage on one’s personal computer can be viewed as an extension of 
one’s long-term memory capacity. Similarly, the currently active file on one’s computer screen 
can be viewed as an extension of working memory. Moreover, some basic and control-level 
memory processes can be enhanced by tool use. For example, project management software can 
make human problem solving more strategic and intelligent. 

In applying this model to group behavior, Hinsz et al. (1997) began by defining group 
information processing as “the degree to which information, ideas, or cognitive processes are 
shared, and are being shared, among group members and how this sharing of information affects 
both individual and group-level outcomes” (p.43). Shared refers to the homogeneity and overlap 
in task-relevant prior knowledge that exists within the long-term memories of group members. 
Sharing relates to processing activity that takes place within group members’ working memories, 
leading to modification of shared knowledge. Hinsz et al. review a large social cognition 
literature that views group effectiveness as a function of how information is distributed among 
members plus the extent to which that information is shared, combined, and changed during 
processing. For example, in group discussions, individuals tend to actively share ideas that are 
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held in common with other members, while ideas representing a single, minority viewpoint are 
less likely to be shared. Moreover, status differences among group members influence degree of 
sharing. Hence, "groupthink” (Janis, 1982) tends to reflect, even exaggerate, the beliefs and 
thinking tendencies of a dominant social M in group.” 

Distributed cognition. Although Hinsz et al. (1997) use the information processing model as an 
analog for describing group process, they do not extend this analogy by defining group 
counterparts for human memory structures and processes. Yet an adequate information 
processing model for groups would appear to require the reconceptualization of concepts such as 
long-term and working memories, attentional processes, and executive control to better describe 
the collective case. 

This reconceptualization is taking place to some extent within the literature on “distributed 
cognition” (e.g., Pea, 1993; Salomon, 1993; Salomon, 1996). The typical unit for analysis in this 
literature is the “activity” (Pea, 1993), such as a single discussion within a meeting. Hence, a 
system of distributed cognition is to some extent a transient artifact of a particular situation — for 
example, the collection of capabilities and ideas mobilized for a particular NISE working-team 
meeting. However, distributed information processing theory supports developmental research 
on groups, since a work group can be studied by examining and comparing samples of group 
memory processes at different points in time. As a working group matures and many of the same 
people come together repeatedly for group activity, the capacities and ideas that the group 
represents would be expected to change and adapt themselves to one another and to activate and 
use each other in increasingly more organized ways. 

Using a hypothetical group activity as an example, Salomon (1996) depicted a group’s long-term 
memory as the salient, activity-relevant information network that is distributed across group 
members’ individual long-term memories. Within this network, some ideas (represented as 
nodes) were viewed as unique to single individuals; other nodes represented shared information 
held in common by two or more members. Shared information thus provides potential bridging 
relationships among otherwise privately held knowledge stores, linking all data in collective 
long-term memory. (Note that drawing a connection between shared knowledge , from the 
information processing perspective, and finding common voices andjboundary concepts , from the 
situated cognition one, is itself an example of finding common voice between two research 
communities.) 

Extending Salomon’s framework, the concept of a group’s distributed working memory would 
represent those thoughts (nodes) from distributed long-term memory that become active within 
any individual’s attention during the group’s discussion. Obviously, only ideas that are shared 
(discussed) by one individual (and attended by others) have the potential to modify a group’s 
collective long-term memory, bringing members into greater conceptual alignment with one 
another. However, as Hinsz et al. (1997) described, an individual’s tendency to share a private 
thought with a group would be influenced by such factors as the individual’s perception that the 
thought is also held by others, plus the status of the potential speaker. Sharing would also be 
influenced by executive, or metacognitive, control, such as might be exercised by a chairperson 
or by unspoken normative rules for discussion. For example, the person wanting to share might 
not be called on to speak or might not want to dominate conversation further. 
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In order to operate effectively, work groups must exhibit metacognitive intelligence. 

Metacognitive intelligence has several aspects: it represents an information processing system’s 
awareness and understanding of its own operations, as well as its ability to regulate and control 
those operations. In traditional information processing theories, and in traditional work 
organizations as well, metacognition was a job delegated to higher-level ’’executive controllers" 
responsible for overall system performance. Executive controllers schedule and delegate tasks to 
lower-level "processors" and monitor overall system performance, making adjustments when 
necessary. However, modem distributed theories of information processing, as well as 
enlightened theories of organizational management, have recognized that metacognitive 
intelligence is shared among various members of a system. Just as group memory is distributed, 
so is executive control, as members assert themselves into the checking, monitoring, and 
strategizing responsibilities that might officially belong to a group leader. In fact, distributed 
processing theory suggests that it may be impossible to truly centralize executive functions, since 
members possess metacognitive intelligence and thus will automatically monitor their group's 
functioning, can choose to cooperate or not with a named leader, and may even usurp the leader's 
control when necessary. 

Creating products. The group information processing framework described thus far remains 
incomplete in an important way. Although we have described mechanisms whereby knowledge 
is shared and integrated, we have not yet said anything about how groups produce product 
outcomes. An adequate theory must specify the routes knowledge takes in becoming a product. 
Smith (1994) suggested a framework for discussing these routes. It identifies three classes of 
information produced by groups: tangible, intangible, and ephemeral. Tangible knowledge is 
divided into target products that represent completion of a group’s task and instrumental 
products that merely support the group’s work. Intangible information represents individual 
thoughts, some of which are shared by others (in the sense of distributed), while other 
information remains private (owned by one individual). Ephemeral products , such as unrecorded 
conversation or temporary board sketches, exist only briefly. We have modified Smith’s 
taxonomy slightly as shown in Table 1. 

Listed at the bottom of Table 1 are forms of tangible knowledge , including the target products 
(e.g., conceptual papers, team reports) that provide evidence of goal attainment. En route to the 
production of these target documents, groups typically produce instrumental products (e.g., graphs 
or data matrices) that provide input to their thinking and problem solving but are not intended to 
stand alone as target products. Another important type of tangible information is the ephemeral 
product. Ephemeral tangible products are transitory representations, such as whiteboard diagrams 
or lists on a flip chart, that are produced as byproducts of meetings. Ephemeral tangible products 
support negotiation among group members and are typically discarded after use, though other 
products might incorporate ideas developed with them. 
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Table 1 

Types of Knowledge Produced in Working Groups 



Type of Knowledge 


Example 


Intangible 


Ephemeral private 


Unspoken fleeting ideas 


Ephemeral shared 


Personal mental models 


Permanent private 


Conversation 


Permanent shared 


Overlap of mental models 


Tangible 


Ephemeral shared 


Whiteboard diagrams 


Ephemeral private 


Temporary meeting notes 


Instrumental shared 


Minutes, data tables 


Instrumental private 


Permanent meeting notes 


Target private 


Individually initiated paper 


Target group 


Group constructed product 



As the top part of Table 1 indicates, some important products, such as team members’ mental 
models of team and task, are in fact intangible products that are achieved largely through selective 
processing of personal thoughts and group conversation. The conversation and thoughts are 
themselves ephemeral until transformed into stable group and individual memories of team and 
task that are retained over time. The transformation of private and shared ephemeral knowledge 
into relatively stable mental models is an important aspect of group processing. 

Yet another important kind of knowledge produced during group interaction is intangible and 
includes both private and shared knowledge. Intangible private knowledge is what is available in 



participants' heads that is not known by others. Group members' private thoughts during meetings 
are difficult to access, so study of private intangible knowledge is somewhat problematic. Much 
shared intangible knowledge is revealed in conversation, since shared knowledge is likely to be 
discussed. Although conversation is an ephemeral form, we theoretically assume it reveals a more 
permanent (though evolving) underlying system of shared group knowledge. This intangible 
shared group knowledge is in fact an important group product. Recent research on team 
performance and group decision making underscores the importance of team members developing 
overlapping “mental models” of their team and task (Cannon-Bowers, Salas, & Converse, 1993; 
Salas, Prince, Baker, & Shrestha, 1995). Although too much shared knowledge can be a liability 
that limits unique and creative contributions, team members’ mental models must be sufficiently 
compatible that they lead to common expectations about other team members and the group’s task. 

Smith (1994) proposed that group processes be modeled by developing a vocabulary for 
describing how information is transformed from one state to another and by identifying typical 
sequences of information transformations, or “conversational phrases.” From this viewpoint, 
negotiation (as the term is understood in the situated cognition literature) is seen as an important 
transformational form made up of phrases for passing intangible (private and shared) thoughts 
through an ephemeral form (such as conversation) in which they are combined and transformed 
to become shared mental models. The routes through which this still-intangible knowledge is 
then transformed into tangible products are not yet known, although this question is important in 
studies of group productivity. However, the negotiation of compatible mental models among 
team members would appear to be an important intermediate step leading to creation of tangible 
group products that represent shared interdisciplinary perspectives. 

Toward an Integrated Theory 

Although a fully developed theory of distributed information processing does not yet exist, the 
emerging framework described above is already providing useful guidance for the study of group 
process in interdisciplinary teams in the NISE. To provide a rough idea of how the language from 
these perspectives can contribute to the description and understanding of group process, the reader 
is asked to imagine that an interdisciplinary group has been assembled to accomplish a generally 
stated research goal, such as "develop a model for analyzing the impact of federal funding policy 
on educational practice." 

Teamwork begins, and each constituent brings to each meeting activity a network of ideas 
representing the individual’s prior knowledge that is relevant to the group’s task. Each individual’s 
network represents thought patterns and knowledge that are characteristic of the disciplines, 
cultures, and communities to which that member belongs. Because individuals have some common 
backgrounds (including their broader institutional context), each individual network overlaps 
(through common ideas) to some degree with ideas brought by other members. This is the group’s 
task-relevant shared knowledge. Also, each individual possesses task-relevant private knowledge 
that is not known by other members and that may or may not be shared by them. 

As conversation ensues, members of the group share (discuss) some of their ideas about the task, 
and they process ideas shared in discussion by others. During early team development, group 
members attempt to develop compatible mental models pertaining to their team and task. Since 
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multiple disciplines are represented, building shared task and team models requires members to 
negotiate language, definitions, and procedures for their work. Even terms held in common must 
be clarified and redefined. For example, the term model itself connotes one set of meanings for a 
mathematician, another for a psychologist, and yet another for a computer scientist. 

If there is a critical mass from one disciplinary area, a dominant rhetoric would likely emerge, 
since shared knowledge is more likely to be discussed than unshared knowledge. This might even 
create an in-group/out-group split that must be dealt with. The group might deal with this split by 
apprenticing out-group members so that they gradually adapt to the in-group rhetoric. For example, 
if there were three psychologists but only one mathematician and one computer scientist, the 
psychologists could instruct the others in their position on modeling. However, this might 
minimize discipline-related contributions from out-group members, which is not a desirable 
outcome for interdisciplinary teams. 

As conversation and perhaps argument proceeds, temporary diagrams or charts (tangible 
ephemeral products) may be constructed to help express technical or conceptual ideas for which 
common vocabulary may not yet exist. A leader and some group members themselves monitor and 
comment on the team’s progress, suggesting meeting formats and communication tools that might 
encourage productive sharing and argument that includes all team members. Such argument is 
understood to influence productive evolution of the group’s distributed long-term memory. 
Successful conversational processes should cause members of the group to (a) adapt their personal 
mental models to better fit various communications received from other group members and (b) 
change others' models to better match their own evolving viewpoint. Understandings thus 
negotiated create compatible understandings of the task and team that become sufficiently aligned 
but not identical among members, since different members are still expected to retain unique 
disciplinary knowledge about different facets of their problem. 

When negotiation of compatible mental models is achieved to some extent, the team is better 
positioned to begin negotiating strategies for creating their goal product (the model for analyzing 
the impact of federal funding policy in the example case). This new phase of activity may involve 
assigning specific tasks to those with special expertise required by the task. A new class of 
instrumental products, such as PERT charts or spreadsheets, might prove especially useful in the 
product development phase. Throughout this phase of group development, factors such as status, 
leadership style, and compatibility of mental models will continue to influence group interaction. 
Thus, the group as a unit should continue to monitor its functions with respect to such factors. 
Although some monitoring may be provided by a leader who centralizes control and decision 
making, such centralized control requires the tacit, if not explicit, consent and participation of 
group members. 

In sum, our developing view of interdisciplinary collaborative research implies that, en route to 
task completion, groups may pass through developmental phases that are characterized by use of 
different conversational forms, tools, instrumental products, and mental activity. Although we do 
not yet know the boundaries of optimal group process for interdisciplinary teamwork, we suspect 
they may differ substantially in different phases of work. For^example, although apprenticeship of 
members into a dominant rhetoric might prove detrimental in early group development, such 
apprenticeships might be necessary and helpful if new members enter in later stages of group 



8 




16 



work. Or, centralization of management and metacognitive control might be more necessary and 
appropriate in some phases than in others. 

Data Source and Context 

The language created by combining the theoretical perspectives discussed previously is applied 
to the analysis of one meeting of an NISE group, the Strategies for Evaluation of Systemic 
Reform (SESR) team. The SESR team is composed of scientists and professionals of both 
genders from the fields of mathematics, astronomy, anthropology, mathematics education, 
engineering, political science, and secondary education. The goal of the SESR team is to develop 
a theoretical model for evaluating large systemic reform efforts funded by the National Science 
Foundation and to advise NSF on the evaluation of such programs. 

We have recorded meeting conversations and taken detailed field notes at all SESR full team 
meetings during the first two years. Meetings are usually two to three hours long. While the team 
originally met formally only six times per year, it now meets every month. We have also collected 
detailed conversational and field data at 2 two-day interdisciplinary working conferences 
sponsored by the SESR team. In addition, we have conducted interviews with team members, 
including several intensive interviews with the team leader and his assistant. 

Examples presented in this paper are drawn largely from observations of the third group meeting. 
This meeting was selected for analysis because researchers and some participants believed it was a 
key meeting in the early development of this group. A brief description of this meeting and a 
summary of the team’s history to this point are provided as context. 

Contextual History Leading Up to Third Meeting 

First two team meetings. At the first two meetings, group members became acquainted and 
struggled to define systemic reform and negotiate the nature of their task in general. The first 
meeting consisted mainly of each team member in turn explaining his or her own view about 
systemic educational reform (SER), with members more experienced in systemic reform providing 
some mentoring of members with less experience. Conversational analysis revealed little debate or 
extensive building by one member on the ideas raised by another. The second meeting began with 
a walk-through of a document that had been prepared by the project assistant to help educate the 
team about its mission and then moved on to a discussion of papers being commissioned by the 
team leader and logistical issues about an upcoming conference being organized by the team leader 
and his staff for the purpose of helping conceptualize the team’s work. In both meetings, members 
expressed a need for further information and material about systemic reform and the systemic 
initiatives (Sis) funded by NSF. Also, meeting activity was solely conversational, without use of 
props or tools such as a flip chart or a blackboard. This period has been characterized as an 
apprenticeship phase of team development. A more detailed description is found in DuRussel and 
Derry (1996). 

Between meetings, the team leader and the project assistant worked to provide educational 
materials that had been requested by the group. For example, the team leader and assistant 
compiled several documents to provide the team with more information about NSF-sponsored Sis 
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across the nation, which were presented at the second meeting. These documents included a matrix 
that summarized a number of major features of all NSF-sponsored Sis across the United States. 
Each row of the matrix represented a separate Systemic Initiative and included 13 entries that 
specified the location of the project, the grade levels covered by the project, who was evaluating it, 
its major goals, and ten other similar items of information. More detailed data regarding a specific 
example project also were supplied. Other between-meeting activity was organization of the 
conference by the leader and assistant, including requesting and reviewing papers from keynote 
speakers, creating the conference agenda, and leading the conference. 

Team-sponsored conference. The conference was designed to bring a larger interdisciplinary group 
(about 30 educational practitioners, evaluators, and researchers) together to work with the team to 
identify questions that evaluators of systemic reform efforts should address. Prior to the 
conference, keynote speakers each were asked by the team leader to provide a "brainstorming" 
paper overviewing their thoughts on evaluation of systemic reform. Papers, as well as other 
informative material, were furnished to all conference participants prior to the conference. 

Over the course of the two-day event, participants were divided into four small groups, and each 
group was given a leader and a set of task instructions. The task, which was the same for all 
groups, was to generate a set of questions that SER evaluations should ask. In each group, a 
facilitator from the University’s Total Quality Management office wrote team members’ comments 
on a flip chart, which was then used for later reference in refining the questions. Also, an observer 
for each group took notes and recorded the final list of questions on a laptop computer; this list 
was periodically printed out to serve as handouts for other conference participants. Each group’s 
questions were included in the conference proceedings generated by the team leader and project 
assistant. These proceedings were distributed to the working group prior to the third meeting. 

Third Team Meeting 

Ten members, including the team leader and assistant, attended the third team meeting. Compared 
to the first two meetings and the conference, the third meeting provided the group with a somewhat 
broader assignment. The team leader’s agenda indicated that the goal of this meeting was to 
discuss and reflect on the conference and proceedings, to refine the questions developed at the 
conference, and to set goals for the group’s Year 2 proposal to the funding agency. But as the 
meeting progressed, it began to differ from the previous meetings in that group members began 
raising and discussing ideas that were not part of the agenda. Also, this meeting was the first one at 
which the group itself created a tangible product: a list of key tasks written on a blackboard. The 
change in character from previous meetings was described by one team member in a postmeeting 
interview: “I felt there was a shift. . . . Because it was like the group had gotten to a point where it 
could do something that active. . . . [Creating a product] was a bit of a turning point.” 

Because the work performed in meetings is largely conversational, and because the productivity of 
that work is largely judged by the content of written products that eventually are produced, our 
studies are attempting to track the emergence and fate of key ideas that are raised in meeting 
conversations, determining their phases of transformation en route to becoming part of (or being 
excluded from) a final product. Understanding why ideas generated in some discussions survive 
and undergo development, and why others are dropped from discussions and/or products, is also of 
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interest. In addition, we are interested in observing the fate of interventions whereby the leader and 
other members of the group attempt to influence the direction and goals of the team. 

Analysis of Meeting 3 



Transforming Ideas Into Products 

In order for the "work" of group discourse to survive and have influence upon future group work, it 
would have to become or be incorporated into a more permanent knowledge form. This evolution 
could happen in several ways. First, aspects of the intangible conversation could become part of 
individual group members' memories and thus part of the distributed intangible group memory that 
is activated and shared at future group meetings. For example, a group member might recall a topic 
of conversation from a previous meeting and attempt to continue that conversation at a subsequent 
meeting. As part of individual memories, intangible shared knowledge also might serve as the 
basis for furthering individual work related to team goals. For example, the team leader might 
"listen and learn" from the group and use ideas in constructing theoretical papers and reports. 
Alternatively or additionally, the knowledge construction that occurs during team conversation 
might be "captured" by the team or staff and transformed into a more tangible group product. For 
example, during a meeting the team as a whole might list their insights on a whiteboard, and these 
ephemeral notes might be incorporated into more permanent instrumental products, such as the 
minutes and next meeting agenda, thus serving as tangible input to subsequent team work. 

Meeting 3 provided an opportunity to closely observe processes whereby a group transforms 
conversation (representing shared intangible knowledge) into an ephemeral product, a list of key 
tasks to perform. We were then able to begin tracking whether and how the knowledge content of 
this product was accepted by and useful to the team and project staff as a basis for furthering its 
work. In addition, we were able to observe how the group accepted and used intermediary 
instrumental products, a conference proceedings and a data matrix, that were designed by the team 
leader and his assistant in an attempt to influence the direction and facilitate the work of the team. 

The Action-Item List: A Product of Distributed Processing 

In meeting 3, the group developed its first distributed product: a whiteboard action-item list (see 
Figure 1). During the final 40 minutes of the 2-hour meeting, a team member initiated this process 
by suggesting that the group brainstorm ideas using the board. The purpose of this activity was to 
help define mission-related tasks that the group might perform. Although this activity was a 
digression from the active agenda item (reflections on the conference), it was supported by the 
team leader, as indicated in his postmeeting interview: 



1 . Distillation of workshop questions 

2. What’s going on out there — descriptive 

3. Look at the assessments & assessment methods already being implemented by NSF SRs 
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4. Look across the SR evaluations to get a picture of methods used and determine “what’s 
missing” and see if these gaps can be filled. 

5. Determine how SR $ are spent — by some categories (by evaluating existing SR eval docs) 

6. Actually DO an evaluation to apply the SESR team’s findings — Develop idealized eval 
designs (in out years) 



Figure 1. Task List Produced During a Working Team Meeting 



I think it was XX who said, “Well, let's write it down on the board. Let's do some 
brainstorming,” which I thought was a good suggestion. I don't know why I didn't think about 
that, I just didn't. Didn't think about putting it on a board. 

// 

So, you know, I don't mind other people's suggestions, in fact I welcome that. 

// 

So I thought the suggestions they came up with were more along the order of what I was 
trying to get out of the meeting. 

As the board discussion progressed, a “scribe” at the blackboard (a team member who 
spontaneously assumed this duty) noted suggestions brought up by the group. We observed that the 
written list accurately paraphrased speakers’ comments, and most of the ideas presented during 
this portion of conversation were represented on the board. 

In commenting on the value of the team-generated list in his postmeeting interview, the team 
leader noted the list exercise confirmed the direction in which he was already carrying the team: 

In terms of looking at the list that was generated, some of the things that I got out of that was, 
was essentially two things. . . . [I]t sort of reinforced what we’re doing, this notion of doing 
some sort of a description of the evaluation of systemic reform and to continue doing that and 
that's something that we have been trying to do. 

// 

And then, there's this notion of actually engaging in doing. . . . So, describing what it is and 
studying what it is as well as trying to do it or design. Which is sort of what the idea was 
initially. So, in a certain sense, I take [the list] as an affirmation of the direction that we're 
taking. 

But regarding the value of this work he also noted a caveat: 

And quite frankly, I don't know if we're asking too much of that committee ... or if they . . . 
know enough to know what the next steps are. That's an unknown to me. 
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Influence of distributed processing. It was evident from comparing the list to the full meeting 
transcript that several ideas that were presented, even discussed at length, during the first hour and 
20 minutes of the meeting were not represented on the list of action items that the group created. 
This showed that a significant part of the conversation was not related to and did not culminate in 
the group's surviving tangible product, raising the questions of (a) why some ideas that dominated 
meeting time were reconsidered and incorporated into the tangible group product while others 
were excluded and (b) whether any portion of the excluded meeting was productive and survived 
as useful knowledge. 

One hypothesis was that ideas that were more widely distributed, having been discussed and modified by 
whole group interaction, might be more likely to appear in a group product since they would be "owned" 
by the larger group. If this were the case, then ideas that were considered in depth by many people in the 
meeting might tend to be placed on the list (unless a result of discussion was to reject them), while ideas 
that were only briefly noted would not be included. In fact, some ideas on the list in Figure 1 had been 
discussed earlier in the team conversation, but other ideas that made it to the list were not raised or 
discussed prior to the brainstorming session and were mentioned only briefly at that time. For instance, 
throughout the meeting various people mentioned the need to find out “what’s out there" in educational 
reform (Item #2), and other people stressed that it was important to know whether current assessment 
methods were adequate (Item #3). However, other points that were included on the list, such as looking 
at resource allocation by category, looking across reforms to “see what’s missing," and doing an 
evaluation (Items 4, 5, and 6) had not been previously discussed. Hence, items 4-6 were less likely than 
items 1-3 to represent a common understanding of the group’s work. 

Similarly, some widely discussed and shared ideas from conversation were excluded. For instance, 
early during the meeting, the group discussed in depth how attitudes are key to reform efforts. This 
idea was raised by one team member, explicitly supported by three team members, implicitly 
supported in statements made by two additional team members, and was not refuted by any team 
member. However, although this issue was developed during the conversation with many team 
members elaborating, it was not reflected on the list of things to do. Thus, topics representing 
widely shared concepts did not necessarily influence the instrumental product upon which the 
group planned to base its future work. 

Idea exclusion: The attitudes conversation. Since shared understanding of an idea as reflected by 
team conversation did not seem to affect whether it was or was not listed on the board, other 
factors must have played a part. One possible influence was the nature of the idea versus the nature 
of the list. The team leader opened the brainstorming session with the question, “What can we do 
that’s practical?” This framework for discussion may have cued participants to bring up only 
action-oriented ideas. More abstract ideas, such as how attitudes might impact reform efforts, were 
not easily restated in practical terms, so they were not included. Of course the attitudes topic could 
have been transformed into a practical action item. For example, the list might have included, 
"Identify or develop assessment methods that measure attitude change." 

Failure to include the attitudes topic did not mean that team members thought the idea was 
unimportant. For instance, one team member in a later interview mentioned that attitudes strongly 
influence reform efforts. The attitudes topic was also included in the minutes prepared by the 
project assistant and team leader after the meeting. Another verification of the topic's importance 
came from a later interview of the team leader, although he used the term "beliefs" rather than 
"attitudes," slightly altering the conversation's meaning: 
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The fact that it didn't necessarily come out on the points, the six points, doesn't 
mean that I didn't attend to it and didn't feel like that observation of reform as being 
beliefs is very, very important. As I mentioned, I've used that comment since then 
several times. I've used it at the Forum 2 in Washington, DC, in talking to people, 

I've used it in talking to people at NSF. . . . It's made me more sensitive as I study 
materials. 

To help determine why a topic that was considered important and highly elaborated by the group’s 
conversation was not translated into a tangible practical action item, we looked in detail at the 
“attitudes” conversation. This issue was raised 30 minutes into the meeting, and the discussion 
lasted for approximately 15 minutes. This segment of conversation is shown in diagrammatic 
format as Figure 2. 

The conversation began when a team member who had not attended the conference noted, 

one part [of the conference proceedings that] didn’t come out real crisp and clear, 
and that’s attitudes, . . . attitudes of people in the system. ... In terms of evaluating 
it [reform efforts] and . . . how [attitudes are] going to get in the way of any kind of 
reform you want” (Box 1 on Figure 2). 

Other team members strongly agreed: 

Box 4: I think he’s hitting the nail right on the head, that systemic reform needs changes in 

attitudes, and if the changes in attitudes don’t happen, systemic reform is not systemic. 

I think that is the central issue. 

Box 5: It would be nice to be able to say [that] our evaluation really addresses some of these 

. . . feelings that people have. 
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Figure 2. Graphic Representation of “The Attitudes” Conversation 
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At this point the conversation turned to the question of the relationship between attitudes and 
student outcomes, generating some debate about whether there is a causal effect or not (Boxes 
8-12). Then, people discussed the wide range of stakeholders — including parents, students, faculty, 
and administrators — whose attitudes all might play a part in affecting reform (Boxes 14-18). This 
discussion about the relevance of attitudes to systemic educational reform (SER) was shifted into a 
slightly different topic when the team leader posed the following question to group members, “I’d 
be interested in this group. . . . How many of us really believe that we need to have systemic 
reform?” thus moving the conversation away from the discussion of attitudes. 

Because the actions of the team leader had the effect of redirecting the conversation, it is 
instructive to consider what the team leader recalled thinking during this time: 

I don't remember everything that I was thinking at the time, but I thought, yes, that's 
interesting. At the same time I find there’s a tension in my mind between talking about 
reform and talking about evaluation of reform. 

// 

I want to keep the perspective that what our group is really to look at [is] the evaluation of 
systemic reform. You can't separate those two things, but I do think you can ... be sure that 
the focus is not so much on describing just what reform is and trying to say, now how do we 
create reform, versus putting more of the emphasis on, well this is what people are doing in 
reform, how do we know what they're doing of value, of worth? 

// 

As the orchestrator of the meeting, I also . . . was trying to think about . . . everybody 
participating, trying to think about I want to get something out of this meeting, I don't want it 
to get too much into discussions which might be (reflective pause) . . . maybe that's a 
problem. But I am somewhat goal oriented. 

Thus, we see that the team leader's question served to redirect the conversation about attitudes. 
There was doubt in his mind regarding the productivity of the discussion, and he recalled viewing 
the discussion as pertaining to the issue of defining reform as opposed to the team’s task of 
deciding how to evaluate it. However, no thought was evident regarding the possibility of 
refraining the attitudes discussion in terms of practical actions. Thus, the team leader possibly 
contributed to the team's premature dropping of the attitudes idea before it was developed into 
action form by shifting the conversation away from it. However, interview evidence and minutes 
already cited indicate that the idea continued as part of the group's distributed long-term memory 
and that the team leader in particular continued to reflect about the attitudes conversation long after 
the meeting. Its absence from the team product was therefore not a dropping of the idea in this 
case. 

Nevertheless, the fact that this idea was retained does not eliminate the possibility that seeds of 
good, actionable ideas might be dropped inadvertently from team products as well as team 
memories, and analysis of the attitudes conversation provides a key to the dynamics of how that 
might be prevented. During the time (approximately 35 minutes) between the attitudes discussion 
and the list-creating session, the group would have replaced the attitudes topic from their short- 
term memories. It is possible that the action-item conversation, which was geared toward 
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practicalities, failed to cue retrieval of more abstract ideas from the group’s distributed long-term 
memory at the time the list was created. 

A procedure that would protect against the possibility of oversights of this nature would be to 
create a group product throughout the meeting. Moreover, to encourage productivity, teams might 
develop a practice of trying to rephrase all discussed ideas in action-oriented terms. 

Group Concerns Regarding Their Role 

Recent research on natural groups has highlighted the importance of role definition for group 
productivity (e.g., Berteotti & Seibold, 1994). Issues associated with defining the roles of members 
can also affect group morale. Given the potential importance of this topic, it is interesting to note 
that two discussions concerning the appropriate role for the group occurred both immediately prior 
to and during the list-making portion of the meeting, although this topic also was not explicitly 
reflected in the team’s list of action items. 

Role-related discussions. The first mention of roles came before the list exercise when one 
member introduced the topic as follows: 

I’d like to unite a lot of things that were said. I think this idea of whether we should be 
doing [X] is a secondary question. If we ask what is our role in this, why do we exist, I think 
one of the answers is that we want to be a think tank for the NSF. And the NSF has a . . . 
central question. Is this valuable, all of this money that we [they] are spending [on systemic 
reform]? 

The role issue was not discussed further at that time. 

Then, during the list-making conversation, the same team member reopened the question of what 
the team’s job is and expressed the following concern: 

The problem will be if we are remaining too aloof to be useful. At some point, you really 
have to get in the trenches and mess around in the mud in order to get something meaningful 
out of it. 

Another member said a question she had had since the beginning was, “To what extent is this a 
working group?” 

The team leader gave his opinion that the group was primarily advisory, but added: 

At the same time, I never would put a stop to it if anyone wanted to get their hands dirty. . . . 
The thing is, I realize that you all are here volunteering your time to a certain extent, and so I 
did not want to abuse that. ... I respect that. 

He then stressed the immediate team goals that he had in mind: commissioning papers, tuning 
current instrumental products (e.g., the matrix), and distilling questions from the conference. There 
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was no further explicit discussion of the roles issue and the team leader seemed to assume that the 
issue was closed. 

However, continued concern with the team’s role did seem to persist implicitly in the entry of a 
list item suggesting that the team consider actually conducting an evaluation themselves (Item #6). 
Four team members expressed support for this idea and only the team leader expressed hesitation. 
However, the explicit suggestion to give further time to considering the team’s role was never 
made. Given that group members appeared to remain concerned or confused about their role, and 
given the potential importance of defining roles for group members, why was this not considered 
an item for further action? 

Factors influencing lack of action on role-related topics. Since the discussion occurred while the 
list was in progress, forgetting was not a possible factor. However, as mentioned previously in 
context of the attitudes discussion, the framing of the list-making part of the meeting might have 
had an influence. The team leader's direction to the group was to think of practical actions. 
Although the concern over team role could have been restated in practical terms (e.g., “Have 
meeting devoted to reaching consensus about the proper role for this group.”), this did not occur. 
Such translation would be evidence of the group’s awareness that a shared mental model of the 
team was needed, a level of awareness we believe would indicate widely shared metacognitive 
knowledge about an important factor in team cognition. That this awareness did not exist was 
evidenced by such actions as the scribe’s not writing the roles issue on the board and no one 
challenging the exclusion. As a result, the topic was not reflected in the product that supposedly 
summarized the things the group felt were important “next steps,” though the content and 
dynamics of the conversation indicated that the team had not reached agreement or closure on its 
role. 

The role issue was also circumvented by the minutes of the meeting prepared by the team leader 
and his assistant. The meeting minutes stated, without elaboration, that the group reiterated its 
focus as a “think tank.” Thus, the minutes represented the team role question as resolved, although 
conversational analysis of this portion of the meeting suggested that this issue was far from 
resolved in the minds of some team members. 

A subsequent interview with the team leader confirmed that he was viewing the SESR team as an 
advisory board rather than a working group and that he did consider the team role question to have 
been resolved at the meeting. He viewed the push for active involvement as primarily coming from 
one team member. However, this interview also indicated that the team leader had reflected and 
was continuing to reflect further on the feasibility of making the team more active. But doubts 
about the ability of the team to perform an evaluation were expressed: 

But . . . maybe there is something that we could do in terms of constructive evaluation, but 
then, I . . . also see the complications. . . . That is you're going to have this group that’s going 
to do an evaluation, that requires some significant amount of effort. . . . That requires 
somebody to go out and gather data, that requires somebody designing instruments, that . . . 
etc. That requires buying some of their time and if we start buying their time, how much of 
that time, or how much money would the group have ... for that sort of thing? And how 
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much of that could be done on a voluntary [basis], so I start to see ... all the barriers ... so 
then I take that into consideration also. But there could be things that could be worked out. 

The team leader's assistant shared similar doubts: 

The idea of this group actually doing an evaluation, I think, is very unrealistic. For a lot of 
reasons. Just access to information being one of them; you just can't necessarily go out and 
gather the type of information that’s required to do this. And designing an evaluation, I think, 
is different, and I think that’s something that the group maybe could do, but . . . that’s 
something that would require an awful lot more time than has been put in so far. ... I don’t 
think that there's not a willingness to give that time, but I’m just not sure that the time 
required is available to us from everybody in that group. 



Following this interview, possibly prompted by it, the team leader placed "Discuss role of the 
team" as the third of four agenda items scheduled for the next meeting. That meeting has now 
taken place. Due to time constraints created by focusing on other items, this item was not in fact 
addressed at any length. 

Individual versus distributed idea processing . The “team role” issue also illustrates how individual 
processing can dominate over distributed processing in a group setting. Clearly, the team leader 
and the project assistant held their own preexisting schemas for what the meeting would cover. 
That preexisting viewpoints affect cognition is a well-established finding of cognitive research 
(e.g., see Brewer & Nakamura, 1984). If the team leader’s original agenda is viewed as a schema 
for the meeting development, it can be shown that the minutes were framed in accordance with the 
agenda, even when discussions deviated from the planned schedule. For example, in the minutes, 
the “attitudes” issue was described as something missing from the proceedings document. Other 
ideas, such as questions regarding the group role, were not included in the team leader’s schema, 
and they were filtered out or changed in the minutes. For example, the minutes gave little 
recognition to team members who worried about being too theoretical. The minutes showed other 
evidence of how the team leader and assistant filtered information, causing their individual 
cognitions to mask or filter some components of the group’s distributed cognition and amplify 
others. The distributed cognitive structure developed at meetings is necessarily filtered through the 
individual cognitions of the project assistant and team leader at the time some products, such as 
minutes, are created. Of course this occurrence is completely appropriate to the extent that one 
duty of the team leader is to provide direction, which possibly means elevating some suggestions 
while devaluing others. However, in using team products in research to judge the decisions or 
knowledge developed by a group, we note that some products may be colored by the ideas of a 
team but very strongly represent views of a leader or other product developer, which may even be 
in opposition to widely shared group beliefs. 

Using Products as Tools for Furthering Group Work 

The above discussion explored the degree to which an idea that is shared in conversation might 
contribute to the generation of group products such as task lists, meeting minutes, and other 
documents. However, these instrumental products are not only outputs but also inputs to the group 
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interaction in that they are intended, and sometimes thoughtfully designed, as tools for furthering 
group work. One question, then, is how these products are used to facilitate group work and 
whether there is a link between the way a product is developed and its usefulness. 

There were three main documents prepared by the team leader and project assistant for use by the 
group: a matrix listing components of various reform efforts (previously described), an 
accompanying case study giving an in-depth look at a particular reform; and conference 
proceedings summarizing lists of questions generated by the SESR conference. The matrix and the 
case study were created after the first observed meeting, during which several people expressed the 
need to know more about existing reforms. The conference proceedings were written by the project 
assistant after the conference. The matrix and case study were distributed before the second 
meeting, while the conference proceedings were distributed before the third meeting. 

The case of the matrix. Meeting conversation indicated that the matrix and attached case study 
documents, intended as tools to help the team, were not meeting the needs of some members. At 
one point a team member said that “the matrix helps a bit, but you’ve got to get a feel for [whaf s 
going on with reforms]. ... I don’t have a good feel for the range.” The team leader responded, 
“We have collected [and] accumulated information, and what you have here on this matrix is really 
a condensation of stacks of material.” Although team members had looked at the matrix and case 
study and had requested such material, it did not appear that they had been able to use them. 

During the meeting, suggestions were made about how to improve the documents. For instance, 
someone suggested that a narrative be added to the matrix to give a better feel for what the 
information means. 

This suggestion raises the question of why the provided instrumental documents were not useful to 
the team. One possibility pertains to the fact that, while the perceived need for this type of 
information was a highly shared cognition, the process of creating the information was handled 
primarily by one "processor," the project assistant. A request from a diverse group actually 
amounts to a large number of slightly different requests. These multiple requests are filtered 
through the individual cognitions of the staff, so the product is shaped particularly by those 
individuals. When the instrumental product is then introduced at a subsequent meeting, group 
members will inevitably see gaps in the material and suggest modifications. These modifications 
are again filtered through the team leader and project assistant, so the cycle repeats itself. 

In group systems, it is perhaps inevitable that some shared desires for information must generate a 
series of actions that will only gradually lead to development of a usable instrumental tool. 
Unfortunately, this cycle can break down in numerous ways. For example, the SESR team became 
frustrated at not being able to obtain what it felt it had requested. The team staff became frustrated 
at not being able to provide what was requested and may have believed the team members were 
not valuing documents or using them properly. This reaction seemed evident at the meeting when 
the team leader used the matrix to answer a question about reform in a particular state, seeming to 
model use of the document for the team. Frustration was also reflected in comments made during 
the follow-up interview with the team leader: 

I thought we provided some information, but then the group comes back and says that that's 

not detailed enough for them. . . . Apparently we haven't done enough because when we give 
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them a matrix, then they say, well that's not enough. So, it's more like we'll know it when we 
see it. But we haven't seen it yet. 

When one also considers the broader institutional context in which work is performed, the situation 
becomes much more complicated because there are additional, often conflicting, requests for 
information that are placed on team staff. Since resources are limited, there is a need to integrate 
requests and create documents that serve multiple purposes. For example, NISE teams must 
produce deliverables by certain target dates, so there is sometimes a perceived need to package 
what are intended as instrumental documents as if they were end products. Packaging may involve 
editorial and polishing processes that slow the turnaround and completion of needed instrumental 
documents, greatly limiting their usefulness as tools for work. 

An interview with the project assistant revealed sources of frustration and difficulty imposed by 
broader institutional contexts directly impacting the matrix and case study documents: 

I think that has been probably one of the greatest sources of frustration for me . . . some of 
the products that we've produced to educate that group and how little usefulness we've found 
in those. . . . Thinking back on it now, I think part of my problem is that I was writing for two 
different purposes. Particularly creating that matrix, I was thinking as much or more about 
the Systemic Initiative evaluators and how they would read it ... as I was about how it would 
be useful to our group. 

Regarding his obligation to the systemic evaluators, the project assistant stated: 

I feel a responsibility. We've asked them to provide us with some materials about their work 
. . . they get a lot of requests for this and that type of information, it comes from NSF and it 
comes from within their states and from third party people like me. ... I promised them 
something when we made those requests. And I want it to be useful to them. 

In addition to considering what information processing and contextual difficulties might be 
associated with having central staff prepare instrumental working documents for interdisciplinary 
teams, it is necessary to consider that there may be significant educational value associated with 
the process of actually preparing documents as opposed to merely receiving the results. In the acts 
associated with sorting through material and creating the matrix, the project assistant and team 
leader no doubt got a good feel for the information because they were exposed to all of it. In 
contrast, the working group saw only the end product, which was limited in what it was able to 
convey. Perhaps, as the team itself speculated elsewhere in meeting 3, it is necessary to work with 
information directly in order to understand it. We speculate, then, that some "hand-on" "in-the- 
trenches" involvement is necessary for understanding, even though working groups may have no 
desire or time to participate at this level. 

The case of the conference proceedings. The case of the conference proceedings is an example of 
how the group responded to an instrumental product that was thoughtfully designed by the team 
leader for the purpose of advancing the conceptual work of the team. At the start of the meeting, 
the team leader asked the group for their reflections on the conference and proceedings. The 
agenda indicated that this reflection process was to lead into a discussion of activities that would 
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help refine the questions listed in the proceedings. However, during the meeting, the group shaped 
the meeting differently. There was not much response to the team leader’s original question about 
the conference. Instead, one of the team members (who had not attended the conference) shifted 
the conversation to an animated discussion of attitudes and systemic reform. Certain suggestions 
for the conference proceedings were raised, but the group itself did not discuss the conference 
questions at all. Instead, they suggested in Item #1 of their action-item list that refinement or 
“distillation” of the questions should occur. Whereas the agenda might have suggested that the 
group perform the distillation, the group called for someone else — as yet unspecified — to do that 
task. In this conversational segment, the group as a whole seemed to follow an agenda created by 
the group, not the one presented by the team leader. The product (conference proceedings) was 
acknowledged as a potentially useful document, but the group as a whole seemed to decline to use it. 

Instrumental products: A summation. There are several interesting findings in our look at the 
utility and use of instrumental products. First, at the time of this writing, an instrumental product 
created by team staff at the request of the team had not yet been helpful to the team, although 
further refinements were requested that presumably could make it more useful. Second, the team 
declined to make use of an instrumental product (the conference proceedings) designed by the 
team leader and intended as input to the team’s conceptual work, although the need for further 
work with that product was proposed by the team's action-item list. Finally, an instrumental 
product (the action-item list) actually created by the team to crystallize its thoughts about 
important tasks to accomplish did not include two important ideas discussed at the meeting. These 
observations suggest that, while significant time has been spent in preparation of instrumental 
documents, these documents have not captured all important aspects of conversation and as yet 
have not been utilized as foundations for furthering team work. 

The incompleteness and nonuse of instrumental team products reiterates the questions of what 
influences effective use of such products and whether the distributed nature of product 
development might affect their usability. Distributed cognition is reflected in many people’s 
contributing to the conceptualization and development of a product. A reasonable hypothesis is 
that an instrumental product that emerges from the group as a result of distributed cognition could 
fit their needs better and might have more people invested in using and further developing that 
product. Such a product would contrast with ideas or products that are developed primarily by the 
leader or a small number of people and given to the group to use. For example, products such as 
the matrix and conference proceedings might be viewed as having a somewhat low level of 
cognitive distribution, since they were created primarily by the team leader and assistant (as the 
authors), and secondarily by the group itself (as the inspiration or as providers of input). The list 
from the whiteboard, on the other hand, had a highly distributed nature, since it emerged from 
interactions of the entire group. However, in order for the whiteboard list to be useful to the team, 
it must be presented again in some form for the team to process further. For example, the leader 
might call a meeting to further discuss and refine the team's action-item list. But, unless the team 
leader brings the unrefined whiteboard list again to the table, which has not yet been the case, the 
team will not have further opportunity to build on its product. Since subsequent team meetings 
have now occurred, we have already observed that the unrefined task list did not appear again, was 
not discussed directly, and in this sense did not serve as direct input to the continued work of the 
team. It is the team leader's prerogative to choose not to use the list in this way, and apparently he 
has made that choice. 



22 




30 



In context of this choice, we note that, following his first six months of work with the SESR team, 
the team leader was struggling with how best to manage and utilize his interdisciplinary group to 
accomplish goals. 

How best to use that group . . . that definitely requires some effort to think about and 
probably needs to have a little more effort than what is being done in terms of thinking about 
how best to use it, because I think the people are very, very willing. 

// 

But then I also am . . . not just thinking about what we need to be doing but also . . . how to 
manage the group. . . . The management of the group requires some effort along with the 
achievement of the task, and those don't always necessarily overlap. 

Then What Good Is the Team? 

An important question begged by this analysis is whether the interdisciplinary SESR team has 
value for the NISE, especially with respect to its mission of developing knowledge and theory that 
could inform NSF on evaluation of systemic reform. Interviews with the team leader and project 
assistant indicated that the team is serving two primary functions. First, both see the team as a sort 
of surrogate audience. For instance, the team leader offered: 

I value the group. ... I think that in many senses of the word they're like a thermometer of 
how people think about systemic reform — and, quite frankly, [people] who don't know . . . 
anything about systemic reform, although they're probably a little more knowledgeable than 
most. So I think that has come out of the group . . . where people can say, what do you mean, 
what do you mean, what do you mean? And that's been very valuable. 

The project assistant reiterated this idea: 

One of the audiences that we ought to be thinking about ... is an audience of people like the 
people in our group when they started, that don’t really know a lot about what systemic 
reform is. . . . One of the real valuable parts of having that group to me is to have that set of 
eyes looking at what types of products we have produced or will produce. 

Although serving as a surrogate audience is a useful team role, we observed that a more significant 
and somewhat surprising function is currently being served by the team. There is evidence that it is 
in fact contributing substantively to the construction of a new evaluation theory, which is primarily 
taking place within the team leader's head. The following excerpts from our interview with the 
team leader provide evidence, not only of the beginnings of creative and reflective knowledge 
construction with potential theoretical and practical importance to systemic reform and its 
evaluation, but also of the team's meaningful involvement in that effort: 

[The] group has provided me . . . some context that I've used. For example, this week I have 
an arrangement to talk to [Participant A], who's a mathematician (describes a personal 
collaboration now evolving pertaining to a universitywide systemic reform in teaching of 
quantitative reasoning). 
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// 

[Participant B] is another who has given me a lead on [Participant C], who was at the 
Institute for Poverty and has written identification problems in social science which is an 
economic model for looking at evaluation . . . and I think that there are prospects for looking 
at economic models. 

// 

[And Participant D] was the one at the conference who mentioned the epidemiological 
models ... so that has led me on that strand and so I've actually gotten a book from CDC . . . 
and it talks about marginal analysis versus incremental analysis and provides a distinction 
between those. . . . That book, I think, would serve sort of as a model of . . . evaluation in 
education, its philosophy and terms, in about six chapters, so . . . that's a lead I'm following 
that came out of having [Participant A] there, who is really a fine mathematician in applied 
mathematics. 

// 

And that [Participant E], I have the highest regard for him, ... his thoughtfulness I might 
benefit from just practically, and I've referred to his study on chemistry (describes study). 

// 

Then [Participant F] the rocket scientist, his analogy that we're really looking at galaxies 
rather than experiments, . . . that is something that I've lived with or that has been very 
helpful to me. 

// 

Then [Participant G] view, his view that, as he looks at epistemology, and as he looks at . . . 
this "belief," the notion that reform is really belief, I think he has a sort of a sense of reform 
that is very, very influential. 

// 

And then [Participant H] brings in the idea that ... all this money's being devoted to systemic 
initiatives, how could that money be used otherwise? Because he really views that it's been 
taken away from the research community. 

// 

. . . and that [Participant I] who was there for the first time, I thought raised ... an 
interesting point of view . . . thinking about the classroom. 

// 

So, I can go around and see each one of those has really contributed to my thinking and I'm 
not sure ... if they understand how I appreciate what they've done. . . . So, definitely having 
this interdisciplinary group has been important to me and broadened perspective, and it 
would not be the same [without them] and this book would not be sitting in my lap now. 
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Conclusion 



This paper interpreted a critical group meeting in the early development of an interdisciplinary 
working team. The interpretation employed language and ideas from an emerging distributed 
cognition theory of interdisciplinary collaboration that incorporates concepts from situated 
cognition and information processing theory. 

We first addressed the issue of alignment between group conversation and a product actually 
produced by the group, a brainstorming list of tasks to accomplish. Meeting conversation is taken 
as evidence of the degree to which ideas represent cognitions that have been more or less 
processed by group discussion (that is, ideas that are developed and shared by many team 
members). Highly distributed shared cognitions were not necessarily more likely than less 
distributed ones to be included in the tangible product that was the culmination of the work 
performed at the meeting. In fact, entire segments of seemingly important conversation were 
omitted. 

Generally speaking, we drew a number of tentative conclusions about why such omissions 
occurred. It is possible that, when products are created at the end of a meeting, group memory may 
not recall ideas that are discussed at length during earlier parts of the meeting. Another factor 
explaining omission is that the nature of products and the nature of certain ideas may not be 
matched in form. For example, discussions of abstract ideas about group roles or attitudes are not 
easy to translate into practical items of action, such as those that might appear on a to-do list. It is 
possible and perhaps desirable to reframe important abstract ideas as action items so that they will 
more likely become part of a team's work agenda. In the example discussed in this paper, issues 
about the group's role might have been listed as "Have a meeting devoted to deciding what group's 
role should be." However, turning abstract ideas into action items may require a level of 
metacognitive awareness that might not be present in meetings, especially early in a group's 
development. 

Leadership style, meeting formats, normed practices, and tools might facilitate the translation of 
good ideas into action items and ensure that they become part of meeting products. However, 
unless these meeting products are reintroduced at future meetings, their creation and existence do 
not insure later building on the ideas contained in them. In the case of the team we observed, for 
example, a product created was not later introduced by the team leader, who chose instead to have 
the team pursue another agenda. 

Good ideas that do not get included in products at meetings are not necessarily lost and can reenter 
the conversation and influence later team work if they continue on as part of group memory and 
are later deliberately introduced into team work. For example, an issue raised in a discussion about 
team roles was excluded from products created from a meeting, but was retained in the memory of 
the team leader and later used. In fact, the meaning and significance of the group role conversation 
might not have become apparent to the team leader until he later discussed and reflected on the 
issue after the meeting. Thus, an idea that appeared neither in the minutes nor on the team- 
generated product did appear on a subsequent meeting agenda (though discussion of that idea did 
not take place due to time constraints). 
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The failure to continue conversations from one meeting to another may give the impression that a 
team is not making progress. However, the executive decision not to continue certain team 
conversations may be based on valid leader concerns that certain directions are impractical or 
inappropriate. When agendas are prepared by a team leader who is exerting executive privilege, 
certain issues are selected or elevated in importance. Even issues that may have emerged through 
meeting conversation might be drastically restated or translated by the leader before being re- 
presented for discussion. This management style can be contrasted with an imaginary case where 
ideas and products that emerge inductively from a team are placed on next-meeting agendas as 
they are prioritized by the team itself, without much in-between processing by a leader. In the first 
case, there is centralized management that minimizes (but does not eliminate) distributed 
processing. In the second case there is distributed processing and a greater degree of distributed 
team management. The style of management we observe in the SESR team is more like the first 
case. 

When ideas are imposed by executive decision, the team has the option of declining to use or act 
upon them. This process appeared to be at work in meeting 3, although it is possible that team 
members were not aware of its occurrence. On the other hand, when ideas emerge from the team, 
the team leader may not embrace them. This process also appeared to have taken place, although 
the team leader may not perceive that he devalued team work. These impasses between team staff 
and the team became apparent in our case analysis through study of how instrumental products 
were used. First, the team requested that an instrumental product be produced that would help them 
understand more about the nature of systemic reform. Although products were produced by team 
staff, the team did not find these products helpful and requested further modifications to them. 

Both team and team staff experienced some frustration as a result. Next, the staff produced an 
instrumental document in the form of conference proceedings and planned a meeting agenda that 
included having the team distill and refine the output from the conference. Presumably this 
discussion would have assisted the team leader in producing a deliverable intellectual product. 
However, the team declined to work on the conference proceedings. Instead, the team carried out 
discussions that led to development of its own action-item list. While the team leader found this 
list to affirm many of the directions they were already taking and has incorporated a few, albeit 
transformed, ideas from the list into subsequent meeting agendas, it does not appear that the list 
will be re-presented to the team for further refinement and work. Thus, it can be said that an 
instrumental product developed by the team was altered, if not declined, by the team leader. This 
was the prerogative of a team leader who thinks reflectively about his management and utilization 
of his team. 

These observations led us to the following hypothesis: To the extent that instrumental products 
desired by an advisory or working team are actually produced by staff that the team does not 
supervise, the producer of the requested products will influence their content and structure. 
Moreover, the producer may be influenced by situational factors beyond the team’s knowledge to 
produce products in a certain way. The result is potential for misalignment between instrumental 
products that the team staff provides and what the group feels it needs to further its work. Such 
misalignments are negotiable, but may result in feelings of frustration on the part of both team 
members and support staff. Another approach might be to encourage such teams to take a more 
direct hand in creating their own instrumental products, which may result in better use of the 
product in team work. This hypothesis carries implications regarding appropriate roles for such 
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groups as well as for the nature of tools that should be provided to them. Active group 
participation might be desirable and, in the case of the team we observed, was desired by at least 
some team members. Thus, it could be useful to have the physical and logistical accessories 
available to facilitate such participation. Construction tools such as flip charts or whiteboards 
might be available during meetings to make it easier for the group to create ephemeral products. 
Meeting practices that encourage creative involvement by team members might be established. In 
addition, group products might be retained from meeting to meeting in an extended long-term 
memory (such as a computer system) so that the group can be given continued joint access to 
them. Moreover, use and development of group products could be encouraged by incorporating 
into agendas the time needed to develop them. For example, an agenda item could be: "Create 
team summary of important decisions made today." As needed, team members could be given 
additional access to data and data handling capabilities. For example, rather than preparing a 
matrix for the team, a subgroup from the team might be charged with preparing the matrix with the 
help of a project assistant and suitable computing, advisory, and data resources. 

The preceding ideas are tentative suggestions representing possible ways to increase the active 
involvement of group members who are willing and to enhance the possibility of product 
development through distributed cognition. In many ways, highly distributed cognition is the 
ultimate in interdisciplinary collaboration, since it represents knowledge that emerges inductively 
from an interdisciplinary group that has not been heavily reinterpreted by a leader, who represents 
a particular disciplinary bias. However, it remains to be seen whether such distributed knowledge 
construction is superior or even equal to the knowledge construction that can occur with stronger 
executive control or when an interdisciplinary team is treated as an advisory board. On the SESR 
team we observed, the leader is himself an experienced, knowledgeable scholar of assessment and 
evaluation who is very open and interested in deeply exploring what other interdisciplinary 
perspectives can bring to the problem of evaluating systemic educational reform. Since other 
members of the team have significantly less expertise in the specialty area of study, it is possible 
that the major integrative theoretical work of the group can and should be performed primarily by 
the leader. To this work the team is contributing importantly and significantly, but the form of this 
important final product must be determined by the team leader. 

Given these observations, the following idea was offered to the team leader as a possible way for 
him to restructure his thinking about team work: The team might be encouraged to pursue at least 
two different types of goals — one group-related goal and one leader-centered goal. Under this 
model, the team might be allowed and encouraged to develop its own interdisciplinary agenda and 
products and to operate more independently with less oversight by the team leader. The possible 
role for the team leader in this work would be to help identify an appropriate product goal for the 
group and provide it with appropriate information-processing supports. At the same time, the 
leader himself could continue to develop his own theoretical work, integrating knowledge offered 
by his team of advisors, in this way, NISE might take better advantage both of this team’s capacity 
for distributed cognition based on its interdisciplinary knowledge base and of the leader’s 
expertise. 
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Notes 



The phrase systemic educational reform (SER) refers to large, programmatic urban, rural, 
statewide, or regional efforts to reform education by mobilizing cooperative partnerships among 
state and federal government agencies, businesses, educational organizations, and public sector 
groups. These groups combine resources and collaborate to align their programs and goals in order 
to have major impact on educational practice and outcome. SERs that are funded by the National 
Science Foundation (NSF) are called Systemic Initiatives (Sis). 

2 Annual meeting sponsored by the research institute in which this team participates. 
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